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5 This is a continuation-in-part of appUcatlon Serial No. 08/228.355. 

filed April 15. 1994. entitled -A DISTANCE LEARNING SYSTEM 
PROVIDING INDIVIDUAL TELEVISION PARTICIPATION. AUDIO 
RESPONSES. AND MEMORY FOR EVERY STUDENT." TTie interactive 
television program is of a ^e disclosed in commonly copending U.S. 
10 Patent Application Serial No. 07/797.298, filed November 25. 1991. and 
In U.S. Patent Nos. 4.847.700. 4.507.680. 4.573.072. 3.947,792. 
4.602.279. 4.264.925. and 4.264.924. the contents of which are herein 
incorporated by reference. 

BACKGROUND OF Tf¥R f NVENTTON 
Interactive television system technologies are currently being 
Introduced into the telecommunications scene. TTiese technologies are 
being applied primarily to provide entertainment and educational 
opportunities to the television industry. Interactive capability has been 

20 developed using one-way systems through the provision of multiple 
paraUel channels of information, related in content to one another. For 
example, commonly owned F^eman et al. patents. U.S. Patent Nos. 
4.264,924 and 4.264.925. disclose interactive television systems where 
switching among multiple broadcast or cable channels based on viewer 

25 selections in response to interrogatory messages allows an interactive 
capability. 

These systems have been enhanced to include memory functions 
using computer logic and memory, where selection of system responses 
played to the viewer are based on the processing and storage of 
30 subscriber responses, as disclosed in Freeman patent, U.S. Patent No. 
4.507.680. 

The benefits of providing interactivity through the use of diflferent 
audio responses was disclosed in Freeman. U.S. Patent Nos. 4,847.698, 
4.847.699, and 4.847.700. TTiese television systems provide a common 
35 video signal accompanied by several synchronized audio channels to 
provide content related user selectable responses. The audio signals 
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produce different audio responses, and to some cases, these are syllable 
synched to a first audio script and to the video signal (such as to a person 
or character on the display) providing the perception that the person's or 
character's mouth movements matched the spoken words. 

5 These prior art systems generaUy provided Interactive capablUty 

through the use of several parallel related infonnation segments, each 
segment requiring a separate channel. What Is needed Is a fully functional 
Interactive program which can be normally received by nontateractive 
television receivers as a conventional video broadcast. What is needed is a 

10 system not limited to receiving multiple paraUel tracks of audio 
transmitted simultaneously. What Is needed Is an interactive system 
vrtiere interactivity Is not restilcted to a question and answer format. 
Because channel capacity Is limited to most transmission systems today, 
vfcat is needed is a system where fiall interactivity is provided to tiie same 

15 bandwidth as currently required by a standard television signal. What is 
needed is a system that todividuallzes the interactive experience through 
personalized graphics video overlay and personalized audio channels. 
What is needed is a system for providing a fially tateractive capability to 
every home, regardless of transmission media. What is needed is a 

20 ^tem for simulcasting both conventional television programmtog and a 
fun interactive program, all withto the same signal. 

pTTMMARY Oy INVENTION 

The present invention can allow each subscriber to a program 
25 received to the home, to receive an todividuallzed and personalized 
experience. A truly innovative aspect of this tovention is the simulcast of 
a Mty toteracttve program allowing participation by subscribers tiirough 
the use of an interactive program box. and yet viewers without the 
toteractive program box can view imtoterrupted normal conventional 
30 programming. Both programs are provided by reception of the same 
standard video signal of normal conventional programmtog. This dual 
functionaUty can be provided by any one of several possible bandwidth 
efficient one-way techniques for sendtog hidden multiple toteractive 
audio segments, graphics data and programmtog control data, the 
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reception and processing of 'which provides a fully interactive experience 
for subscribers. Increased interactive flexibility Is possible through the 
use of external data storage such as CD ROM which can store additional 
audio and graphics interactive elements locally. An important feature of 
5 this new invention is that interactivity is offered as an option, without any 
degradation or Interruption In program content tb users who choose not 
to interact or do not have the interactive program box. 

By creating this more intimate relationship between the viewer and 
conventional programs, a novel exciting kind of television programming 

10 is created. Yet viewing of conventional programs is not affected. 
Individualization to each subscriber comes not only from the 
personalization of audio responses and the memory capability of the 
^tem but video graphics overlays are also personalized further 
enhancing the interactive experience. Furthermore, interactive 

15 programming costs are dramatically reduced since fully functional 
interactive elements can now be carried on normal everyday television 
program broadcasts. 

The interactive innovative features of the present invention can be 
provided to participants either with an interactive program box 

20 ccmnected to a conventional television set or to computer users by way of 
a multimedia personal computer (PC) with an RF demodtilator board and 
decoding hardware and software. 

With the present invention. Interactivity is personalized preferably 
through the use of alternative audio responses and graphics displays. 

25 vridtb. are presented as a function of either a current subscriber remote 
entry or earUer stored subscriber remote entries using the memory 
capability of the ^tem where the system remembers earlier subscriber 
responses and uses these responses in formulating and presenting 
individualized audio and graphics messages to the subscriber. 

30 Personalization of audio can be enhanced through the use of preloaded 
additional audio response segments in the interactive program box and 
stacked audio response segments in time. Further enhancement is 
possible through the use of external data storage devices such as CD ROM 
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for storing interactive audio and graphics elements, increasing the 
responsive capacities of the system of the present invention. 

Unlike previously disclosed interactive systems, the present 
invention no longer solely requires that personalized messages be 

5 presented immediately after the subscriber has made an answer choice 
selection. Tht novel use of trigger points overcomes the limitation of the 
question-interactive answer response formats described in previous 
patents and applicattons by allowing for specialized attention to be 
provided to the subscriber at unknown times throughout the program. 

10 Interactive audio and graphics can show up anytime during the program 
as a function of stored subscriber information. 

This enhanced interactivity is provided through the use of trigger 
points spread throughout the program and identifiable by codes 
eoibedded into the video signal. TWgger points provide designated times 

15 during the program vrhsn the program content is subtlety altered to 
present individual and specialized attention to the particular subscriber 
viewing the television or computer screen. The trigger points are 
essentially markers on the program that effectively trigger macros which 
call for a certain interactive function (e.g.. the playing of a personalized 

20 audio segment and/or the overlay of a graphics screen) to occur using 
stored subscriber selections to questions presented preferably at the 
beginning of the show, or retained from a previous show, and/or in 
combination with a particular pattern of interactions solicited and 
entered during the program. The Interactive program box controller 

25 recognizes the trigger points by decoding invisible codes embedded in 
the conventional video signal. The exact timing of the trigger points are 
unknown to the subscriber in order to make the Interactive elements 
appear unsolicited to the interactive subscriber. In this maimer, the 
interactive elements are presented In a more subtie and less noticeable 

30 fashion thereby increasing significantly the personalized experience to 
home viewers. 

Meanwhile, subscribers either without the interactive program 
boxes or with the box but not authorized to receive the particular 
interactive program continue to receive and hear the standard program 
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audio and view the conventional video without being oflfered interactive 
choices and specialized responses. Therefore, both an Interactive 
program and a nonlnteractlve program are being simultaneously 
transmitted to subscribers. 

Tills high degree of Individual participant interactivity is provided 
to the home television or personal computer through the use of only one 
standard video slgnaL A nonnal television signal has an audio carrier and 
a video carrier. Standard audio, synchronized to the video presentation. 
Is Inserted Into the normal audio carrier, thereby maintaining 
compatibility with any conventional television set In the present 
Invention, unused lines of the video are preferably used for embedding 
additional Interactive response audio channels and graphics and control 
data. Altematlvety. the Interactive audio segments could be provided 
cither serially, one after another, in the audio subcarrler (SAP), or in 
cable frequency guard bands, and/or prestored In memory at the 
interactive program box. In addition, even more audio and graphics 
elements can be provided through the use of external storage devices or 
game cartridges, to any event, with the present tovention it is 
inconsequential whether the toteractive elements are broadcast 
20 synchronously, serially, on separate channels, embedded In the video, 
transmitted during or before the program, stored in external data 
storage, etc. Any of these elements, no matter where they come from, 
are called up by the Interactive box at designated trigger points through 
the use of overlaid logic sent down in embedded codes In the signal or 
25 resident in software at the receiver location, fa effect, a normal 
conventional television program and an Interactive program are broadcast 
simultaneously from a centralized location to a multitude of subscriber 
homes, some of which are equipped to receive the interactive program 
yfrtillc others are not. 
30 The enhanced level of toteracUvlty is provided to fateracUve 

subscribers In their homes through the use of interactive program boxes 
connected to television sets, which receive signals either through cable, 
direct broadcast satellite, television broadcast, or other transmission 
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means. In the present invention, the programming can be provided in 
both analog and digital embodiments. 

It is the ottfect of the current invention to simulcast an interactive 
television program and a conventional television program* all as part of 
5 one standard video signal to each of a multitude of subscribing interactive 
participants. 

It Is an object of the Invention to present Interactive elements 
throughout the program at times unknown to the subscriber. 

It Is an object of the invention to deliver Instantaneously 
10 personalized messages to subscribers in their homes through the 
presentation of personalized audio responses and graphics. 

It is an object of the invention to use an analog transmission means 
for sending multiple Interactive audio signals and data codes ^th a single 
video signal. 

15 It is an object of the invention to use digital transmission 

technology for combining multiple interactive audio signals, a video signal 
and data codes onto a single composite digital interactive video signal. 

It is an object of the invention to remember earlier subscriber 
selections in tailoring personalized graphics and audio responses. 

20 It is an object of the current invention to provide a dual function 

program: a program that is conventional and can be viewed on 
conventional television receiving equipment and also, allow for this same 
program to be received interactively once subscribers have the interactive 
program box. 

25 It is an ottfect of the current Invention to recall interactive audio 

and/or graphics elements firam an external storage device. 

It is an object of the current invention to create an interactive 
television system that can provide demographic Information to a head- 
end location to be used for survey ptuposes or for the creation of targeted 
30 commercials. 

These and other advantages, features and objectives of the invention 
and the manner of attaining them wiU become apparent and the invention 
will be best understood by reference to the following description of the 
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embodiments of the invention in conjunction with the accompanying 
drawings and appended claims. 

BRIEF DRSCRlPTrQN OF THE DRAWIN08 

5 Figure 1 Is an overview of the operations center, transmission 

subsystem and home reception subsystem of the conventional video 
system. 

Figure 2 Is a time diagram showing a representation of trigger 
points and corresponding alternative audio segments, one of which is 
10 selected for presentation to the subscriber immediately after the 
execution of a trigger point function. 

Figure 3 is a diagram of the preferred embodiment of the 
interactive programming box for extracting the interactive audio 
segments and data finom the conventional video signal bandwidth where 
15 . such interactive elements have been embedded into the composite 
interactive video signal at the operations center. 

Figure 4 is a more detailed schematic of the hardware elements of 
an analog four-channel VBI extractor in the interactive program box as 
diagrammed in Figure 3. 
20 Figure 5 is a more detailed schematic of an alternative preferred 

digital embodiment comprising a digital extractor comprising a digital 
decoder/demultiplexer configuration. 

Figure 6 is a diagram of a second embodiment of the interactive 
programming box for extracting and storing the interactive audio 
25 segments from an SAP audio channel in which the interactive audio 
segments are serially aligned in the SAP channel at the head-end. 

Figure 7 is a diagram of a third embodiment of the interactive 
programming box comprising two tuners, one of which is for tuning to 
and demodulating the standard video and audio signal and the second of 
30 which is for demodulating an secondary analog carrier comprising 
modulated serial digital audio segments. 

Figure 8 is a diagram of a fourth embodiment of the interactive 
programming box also comprising two tuners but with a digital 
demultiplexer configuration for demultiplexing the digital audio stream 
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into n parallel digital audio channels, wherein the n parallel digital audio 
diannels are time division multiplexed at the head-end and transmitted 
as a separate digital audio stream. 

Figure 9 is a diagram showing the hardware elements of a preferred 
5 analog transmission embodiment at the operations center comprising an 
inserter for embedding four audio circuits and data commands into a 
single video signal. 

Figure 10 is a block diagram showing the hardware elements at the 
operations center of a digital transmission embodiment comprising 
10 digital encoder/compressor configurations and a digital multiplexer. 

mrr^^w PFffnMPWQw Qg thr preferred n^^opin\ffFf[T 

L Introduction 

The present invention Is a system for simulcasting a fully interactive 

15 program allowing subscribers to interact with the program through the 
use of a remote device (604) and an interactive program box (600) 
connected to a conventional television set (186) or through the use of a 
keypad and personal computer (187). with normal conventional 
programming, viewable by anybody with a regular television set (186) or 

20 computer display monitor (187), as shown in Figure 1. 

While such interactive programming may include a plurality of 
video signals, the Interactive television programming used herein 
preferably comprises a single standard video and audio television signal 
with a plurality of additional audio signals and/or graphics data for 

25 providing Interactivity. The Interaction with the subscribers comes 
primarily by way of selection of one or more linked audio segments firran a 
plurality of audio segments, whereby the selected audio segment(s) are 
chosen as a function of previous user responses. Interactivity is 
enhanced through the use of overlaid graphics displays on the video 

30 *^^ch like the audio responses, also vary according to selections made by 
the subscriber on the remote device (604). Audio segments are used to 
provide personalized responses to subscriber selections. The graphics, 
on the other hand, are used to both quezy the subscriber, preferably at 
the beginning of the program, and also to provide personalized graphical 
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messages to subscribers. The Interactive television program also 
comprises control data for controlling an interactive program box 
controller (178) at the subscriber home. 

Interactivity is further enhanced In the preferred embodiment 
through the appUcaUon of trigger points scattered at various 
predetermined times throughout the program. The trigger points 
correspond to times when interactive events are scheduled to take place. 
These interactive events could be the selection and playing of audio 
segments or the display of graphics. While the choice of particular audio 
segments or graphics is still dependent on viewer selections, the viewer 
selections in response to displayed graphical interrogatoiy messages are 
preferably made during a period at the onset of the program or when a 
viewer first tunes into the program. These viewer selections are then 
utilized as Inputs to macros called up at later times during the program 
by the controller upon the occurrence of the trigger points, identified to 
the interactive program box by unique codes embedded In the video 
signal. Alternatively, there can be an Immediate response firom local 
storage such as RAM. FLASH or ROM. external data storage such as CD- 
ROM, firom the composite Interactive signal or from a series of audio 
segments that can be collected firom both the signal and storage. 
II. System Components 

A General Overylpw 

As shown In Figure 1, the sjrstem uses an Interactive program 
delivery system (98) with any broadcast transmission means Including 
satellite (150), cable (138) or television (142) to deliver the simulcast 
Interactive and conventional program (hereinafter "composite interactive 
program") from a centralized location, or operations center (608). for 
distribution to subscribers in their homes. The program may be 
broadcast live from the operations center (608). For example, live 
sporting events with added Interactive elements can be broadcast firom 
the operations center (608). However, it Is more likely with the present 
Invention that the program will be produced ofif-Une and stored in a 
program storage means (133) at the operations center (608). Likewise, 
distribution of the signal at the receive end can be by any suitable 
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transmission means (166) including a cable distribution ssrstem. fiber 
optic, microwave, etc. In the preferred embodiment, the ^tem 
generally includes: an operations center (608) where the composite 
interactive program is broadcast, the composite interactive program 

5 comprising a single video signal, a plurality of audio channels, graphics 
and control codes: a broadcast transmission system comprising 
transmission equipment for transmitting the interactive video signal by 
any suitable broadcast transmission means and receive equipment for 
distribution to the remote home locations; and at the subscriber home; an 

10 interactive program box (600) for processing the signals and providing 
for the interactivity: a remote device (604) for transmitting to the 
interactive program box (600) subscriber selections: and a standard 
television set (186). Alternatively, the interactive program signal can be 
received, processed and displayed by way of a multimedia peraonal 

15 computer (187) comprising an RF demodulator board, an extractor board, 
and keypad. 

A ma^or aspect of this invention is the individualization of audio 
response to subscribera in the home. Each participant receives 
peraonalized audio responses based on one or more selections on a 

20 remote (604) keypad. Since the system can remember earlier subscriber 
selections, the siabscriber selections may be Input and stored in memory 
at the beginning of the program and used later by the Interactive program 
box (600) in formulating an interactive response. The audio response is 
normally a prepared prerecorded audio response comprising one or more 

25 audio segments. In the present invention, not onty do the audio 
responses provide interactivity, but the graphics displsQrs also can vary as 
a function of previous user selections thus generating more 
individualization of the program. 

At the remote sites, participants interact with the interacUve 

30 program either via a subscriber interface (604) or computer keypad. 
Preferably, the subscriber interface (604) is an Infrared (IR) datalink 
remote device with a keypad or if the interactive program is received by a 
peraonal computer (187), a PC keypad. Depressing a button on the 
keypad (604) provides input to the interactive program box processor 
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(178) which can either Immediately or later be used by the controller to 
trigger unique algorithmic codes to produce a unique and individual 
message from a plurality of transmitted audio segments and/or those 
prestored at the interactive program box (600). The response can be 
directed to the subscriber's current answer selection to an inteirogatoiy 
message displayed on the screen, or may be based oh earlier entered 
subscriber selections through the provision of memoiy and logic in the 
system and trigger points, as described below. 

Multiple audio segments forming the set of suitable responses to an 
inteirogatoiy message are preferabty sent as part of a standard video 
signal. There are a number of different ways to effectively forward the 
necessary audio segments for a given interactive event to the interactive 
program box (600). Different embodiments for delivering these audio 
segments indude digital and analog transmission embodiments including 
serial or parallel paths, as described below. The interactive elements may 
be broadcast synchronously (alternative responses aligned in time), 
serially, on separate channels, embedded In the existing video and/or 
transmitted before or during the program. Audio segments tagged for a 
given Interactive event can be sent to the interactive program boxes 
(600) much earlier than the scheduled event dtulng the program, in 
which case the segments are preferably stored in temporaiy memoiy. or 
the segments can be transmitted concuirently with the event With the 
present invention, it makes no difference how the audio segments re&dti 
the interactive program box (600) as long as they are available for 
selection at the box (600) at the predetermined trigger points. For 
example, the audio segments could also be stored In local external data 
storage (629) such as CD-ROM. 

In the preferred embodiment interactive events occur at 
scheduled predetermined trigger points (900). as shown in figure 2. 
These points (900) correspond to the times when the conventional 
program content can be altered and personalized for those subscribers 
capable of receiving the Interactive signal. Tlie programmer can place 
the trigger points (900) at any time throughout the program. Since the 
trigger points (900) are unknown to the subscriber, the subscriber does 
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not know vrbein they vriB. receive a personalized message. In other -words, 
an interactive response can either immediately follow a corresponding 
user selection made to an Inteirogatocy message or occur at a later time 
coiresponding to a trigger point (900), or any combination of the two. Of 
course, timing of the interactive events should correspond to staitable 
times in the program where branching to interactive elements is sensible 
and does not clash with the program content of the conventional video 
still displayed on the television (186) or other dispk^ monitor. 

As shown in figure 2, at the onset of a trigger point (900), the 
controller will select one of several possible audio (or graphic dispk^) 
responses for presentation to the subscriber. As mentioned above and 
shown in figure 2, some of the audio responses may comprise the linking 
of two or more audio segments. 

In combination with the use of trigger points (900). the present 
invention allows for the viewer to select certain options at the onset of 
the program to suit the viewers* preferences. For example, if the 
program broadcast is a live sports event at an eaxfy trigger point (900), 
the viewer could be queried as to Aether the viewer would prefer to 
receive audio In En^sh, Spanish. French, or pertiaps hear the local 
announcer instead of the netwoxk announcer. Upon the viewer selection, 
the controller branches to the appropriate channd. 

Each trigger point (900) is Identified preferably through the 
broadcast of codes sent as part of the composite interactive program 
signal. The codes prderably include, at a minimum, the following 
information: (1) header identifying the occurrence of a trigger point (2) 
function ID (e.g., selection of audio responses or graphics responses,etc.): 
and (3) corresponding interrogatory message(s). The first bit sequence 
simply identifies to the controller (178) that a trigger point is about to 
occur. Ihe function ID designates the macro or other set of executable 
instructions for the controller (178) to read and Interpret to obtain the 
desired result e.g.. a selected audio response. Alternatively, the data 
codes could Include the actual algorithmic fiuiction for the controller 
(178) to apply. 
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Upon extraction of the codes by the extractor (174), the controller 
(178) reads and Interprets the codes and calls from memory (282. 286) 
either a stored message for display on the screen or a particular user 
selection(s) designated by the trigger point codes. The user selections 
correspond to subscriber answers to a series of Interrogatoiy messages 
preferably presented at the beglzming of the program, i^fter obtaining the 
appropriate user selection(s), the controller (178) reads and performs 
the executable instructions using the user 8election(s) as input(s) in the 
macro algorithm. The result of the algorithm is either a selected audio 
response and/or selected graphics response. The audio response can be 
called frx)m monoiy if it is prestored. caUed from external data storage 
(629). or the controller (178) can command the switch (620) to branch 
to the particular audio channel if the response is broadcast concurrently 
with the trigger point After the selected audio response is played to the 
subscriber, the switch (620) branches back to the standard audio, shown 
at time t. in figure 2. 

As mentioned above, a series of Interrogatory messages are 
preferably presented when the subscriber begins watching the interactive 
program. These Interrogatory messages can be presented in ai^ one of 
three wayB. First, the interrogatory messages can be presented as 
graphics displays overlaid by the interactive program box (600) onto the 
conventional video signal, wherein the graphics data is sent in the 
vertical blanking interval of the conq)osite interactive signaL Second, the 
interrogatory messages are presented as graphics displays as discussed 
above, except the graphics data comes from local storage such as RAM 
(282) or FLASH ROM (286), external data storage (629) (e.g.. CD ROM. 
cartridge, ete.), or a combination of data in the VBI and data called from 
either local (284) or external data storage (629). Third, graphics data 
can be presented in the form of user templates stored at the interactive 
program box (600). 

User selections corresponding to answers to the n successive 
interrogatoiy messages are received by the remote interface (628) at the 
beglxming of the show, stored in memory and used throughout the show 
at the appropriate trigger points to subtlely change program content as 
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the show progresses. Preferably, eadi Inteirogatoiy has a set of possible 
answers. Next to each possible answer will be some identifier 
corresponding to a label on a key on the subscribers remote (604). The 
subscriber depresses the key cozresponding to their answer selection. 
5 This selection is decoded by the remote inter&ce (628) and controller 
(178) . stored in memoiy (282. 286). preferabty RAM (282). and used 
later as required by an algorithm designated at a trigger point 

Graphics can be overlaid onto the video at the subscriber site 
through the use of a character generator (624). as shown in figures 3 and 

10 6^. The character generator (624) is essentially an on screen display 
chip. After embedding the graphics codes into the program video signal, 
the microprocessor (178) interprets the codes and maps the codes into 
the actual bit maps, representing the characters, stored in memory (282. 
28Q. The interactive program box processor (178) then commands the 

15 character generator (624) to overlay a particular character designation at 
a particular point in the screen. 

Alternatively, graphics video can be sent from the operations center 
(608). The graphics video can be created and output firom a personal 
computer (114) to a video mixer (108). as described in Harper et al.'s 

20 U.S. patent application no. 08/228.355. Refeixlng to figure 1, additional 
inputs to the video mixer (108) may include the cmventional video firom 
either Uve cameras or VCR. The video mixer (108) ouQmts a single video 
signal fay switching between the graphics video and program video at 
appropriate times. 

25 A plurality of ^r^hics overisQrs. each of \diich preferably comprises 

an interrogatoiy message and wiOdi are chosen as a fimction of previous 
subscriber selections in response to previous interrogatories, can be 
provided. Ihe method by which a graphics message is chosen is 
substantially the same as the method used to select audio responses. The 

30 graphics oveAays can be selected as interactive responses at trigger 
points as e:q)lained in the preceding paragraphs. As with the audio 
responses, one of several possible graphics overlays can be presented 
during any given time interval. The graphics message chosen is also 
based on the subscriber selections to displayed interrogatoiy messages at 
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the beginning of the program. Hie subscriber selections are called by the 
interactive program box processor (178) which examines these inputs 
according to algorithmic codes to produce the next graphics 
Interrogatory message. 
5 The one-way interactive system also has the advantage of 

remembering subscriber responses and using these responses in 
choosing an audio response, and/or graphics inteirogatoiy message, to 
present to the subscriber. Memoxy branching is a technique of the 
present Invention where the algorithm assembles audio responses and 

10 graphics interrogatoiy messages according to the current and previous 
user inputs. Memoxy branching is accomplished by linking audio message 
segments or successive graphics interrogatory messages together in a 
logical relationship. In this scheme, the Interactive program box 
processor (178) contains logic (preferablty, in the software algorithm) and 

15 memory (284) to store previous subscriber selections and to process 
these previous responses in the algorithm to control future audio channel 
selectioiu as well as future graphics message selection. 
H rtie Interactive Prop-am Box 

Figure 3 shows an overview of a preferred Interactive program box 
20 (600). Figures 4 and 5 show analog and digital embodiments of the 
audio/data extractor (174), a component shown in figure 3. Alternative 
embodiments of the Interactive program box (600) are shown in figures 
6*8. The embodiments represent different apparatus for receiving, 
processing and storing the alternative interactive audio segments which 
25 are received in dlfiferent transmission formats. With these embodiments, 
the interactive systems are no longer solely limited to selecting audio 
from multiple parallel tracks of audio, related In time and content, nor Is 
the Interactive questions*lmmediate answer format, as disclosed in 
previous patents, necessary. Of course, the S3rstems of the present 
30 Invention can still use the questlon*immediate answer format or a 
combination of such format and delayed response via trigger points. Hie 
concept remains the same, l^e., to select audio responses which are 
matched to user selections by some function. However, the current 
invention, as supported by the disclosed embodiments, goes farther to 
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enhance interactivity by providing the interactive elements at times 
unknown to the viewer. 

The interactive program box (600) is a portion of the system that 
can reside in the home of the subscriber or elsewhere, such as at a cable 
5 headend, as described below. If at the home, the interactive program box 
(600) is usually located near the subscribers' television (186). but it may 
be placed anywhere within the range of the subscribers remote handset 
(604). As shown in Figure 3, the preferred interactive program box (600) 
comprises an RF demodulator (616), comprising video and analog 

10 demodulators, an audio switch (620). voice/data extractor (174). modem 
(312). audio memory (702) a controller (178) with associated memory 
(284). Gen-lock circuit (623). character generator (624). an RF 
modulator (622) and an infrared Interface (628) for the subscriber 
remotes (604). An external data storage device (629) and printer (302) 

IS are optional. 

Alternatively, the elements of the interactive program box (600) 
can be Incorporated and provided by a conventional multi-media personal 
computer (187). Preferably, this computer preferably comprises a video 
demodulator board, a k^pad for entering subscriber selections instead of 

20 a remote IR device (604). an extractor board for separating the audio 
signals and data from the conveiitlonal video signal, temporary and 
permanent data storage, a modem (optional), audio switch and a 
processor. Operation of the interactive program system using a personal 
computer (187) is substantially identical to that of the Interactive 

25 program box (600)/televlslon set (186) combination described below. 

In addition, although the interactive program box (600) is most 
likely to be located in the home, there is no reason why the interactive 
program box (600). or a functional substitute (comprising a processor, 
extractor, memory, audio switch. RF demodulator, etc.). could not reside 

30 at an location external to the home, such as at a cable headend. If the 
Interactive program box (600) is located at the headend, the interactive 
program box (600). or equivalent, could receive user selections from the 
home via a wireless RF link, telephone line, cable, etc. Once user 
selections are received, selection of the audio and graphics signals at the 
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cable headend occurs in identical fashion as if the interactive program 
box (600) were located In the home, as described below. The path to and 
from the cable headend can be seen as simply an extension of the cable 
betweei the Interactive program box (600) and the television set (186). 
5 The only possible distinction Is that one Interactive program box (600) 
located at the headend may be programmed to seivlce several subscriber 
homes effectively using a time sharing box among many users. 

In a preferred embodiment, the plurality of audio signals are 
combined with the single video signal along with interactive data and 
10 graphics codes in an inserter (130) at the operations center to form a 
single NTSC compatible video signal. These signals may be analog or 
digital, as described below. 

Referring to a preferred embodiment shown in figure 3. the 
composite interactive signal is received by the Interactive program box 
15 (600) either via cable connection. If the signal is transmitted over cable, 
by way of satellite antenna hook-up for receiving satellite broadcasts, or by 
w^ of a television anteima If the signal is broadcast The composite 
Interactive signal is input into an RF demodulator (616). In the standard 
television mode with no interactivity, the RF demodulator (616) simply 
20 demodulates the conventional video and audio signal and outputs the 
standard audio which passes through the audio switch (620) to the 
television set (186). The video is forwarded to an RF modulator (622) for 
presentation on the television or computer screen. Therefore, in the 
standard mode the interactive program box (600) is slmpty a pass- 
25 through box. The voice/data extractor (174) and audio switch (620) are 
completely bypassed in this embodiment 

In the interactive mode, the video demodulator (616) outputs the 
standard video signal which is transported to a Gen-lock circuit (623) 
and character generator (624) as wells as to a voice/data extractor (174). 
30 At the output of the Gen-Lock circuit (623) and character generator 
(624). the video is forwarded via the RF modulator (622) to the television 
(186) or computer display monitor. The processor (178) preferably 
controls an n X 1 switch (620), the output of which is an appropriate 
audio segment to be sent to the television set (186) for presentation to 
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tile subscriber. Of course, the switch could have more than one output. In 
which case more than one viewer can watch the y^deo on the same 
monttor and each receives Individualized audio response through the use 
of headphones. The processor (178) sends a command to the audio 
5 switch (620) to disconnect the standard audio at the beginning of an 
Interactive segment. The extractor (174) essential!/ reverses the process 
by vrbisii the audio and data signals were inserted into the video signal. 
As explained bdow. the voice/data extractor (174) removes the additional 
audio segments and data that are hidden in the standard video signal. 

10 The data is forwarded to the microprocessor (178) and the audio 
segments get sent eiUier to an audio switch (620) or to temporary 
memoiy (202) depending on vrhtre the instructions teach the segments 
to be forwarded, all of which occurs under the control of the 
microprocessor (178). The microprocessor (178) reads and Interprets 

15 the instructions either broadcast in the data codes or resident in the 
operating software at the interactive program box (600). 

The micnq)iocessor (178) interprets the extracted data as either 
control data, including instructions for switching between voice channels, 
or graphics data for on screen display. If the data is on-screen display 

20 data, the data is preferably prefixed by a command designating the data as 
on-screen display data, as opposed to control data. In the preferred 
embodiment, the controller (178) also examines the control data for the 
occurrence of a twader code designating the onset of a trigger point in 
the program. 

25 If the trigger point codes designate a macro which calls for the 

placement of a graphics display on the video, the microprocessor (178) 
reads the codes, i^epts any graphics data sent from the head-end (608). 
calls for and examines the actual bit maps stored in memoiy (284) or 
external memoiy (629) and designating the Identity of the characters, 

30 and then commands the character generator (624) to overlay particular 
characters at particular points on the screen. Therefore, the graphics are 
preferably generated locally with the bit maps stored in memoiy (284). 
The graphics are selected for presentation either in predetermined 
sequence, through the use of control codes in the composite interactive 
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program, developed when the program was created at the operations 
center (608). or more flexibly through the execution of algorithms by the 
processor (178) utilizing stored subscriber selections to previous graphic 
interrogatory messages. The algorithms are preferably part of the 
operating systems software stored In memory at the interactive program 
box (600). Alternatively, the algorithms coiild be included in the data 
portion of the composite interactive signal. 

The graphics can be utilized to overiay ax^ portion of the screen of 
the television screen. The character generator (624) is locked by a Gen- 
lock circuit (623) which allows for the synchronous placement of the 
graphics on the video. The character generator (624) is preferably a 
standard on-screen display chip which takes incoming video, locks the 
video and superimposes on the video the characters as Instructed by the 
microprocessor (178). Specifically, the character generator (624) is a 
switching system whldh takes the active lines of video and switches to a 
mode of sending the graphics characters for a predetermined time, and 
then switches back to the video when the character is finished being 
written on the screen. 

Because the graphics are generated locally, subscribers without the 
Interactive program box (600) are not be able to view the graphics. For 
those subscribers possessing the interactive capabili^. the graphics can 
be used for both posing Interrogatory questions to subscribers at the 
onset of the program, consistent with the trigger point embodiment, 
posing questions during the program, or used to provide a personalized 
req>onse to previous Individual subscriber selections. 

Preferably at the begixming of the program or when a viewer first 
tunes in. a series of interrogatory messages are presented to the 
subscriber. The subscriber responds to the interrogatory message by 
depressing a button on a remote (604) or computer keypad 
corresponding to an answer selection listed on the Interrogatoiy graphics 
screen. If the subscriber has made a selection using a remote (604). a 
signal is received by the IR interface (628) which processes the signal 
and forwards the signal to the processor (178). The processor (178) 
preferably creates a packet comprising the user selection and a header 
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code identifies the particular interrogatoiy message associated 'with 
user selection and sends the packet to memory (284). Each user 
selection to each Intexrogatoxy is stored in this fashion. These selections 
yrVl be called later in the program at expropriate times when identified 

5 by the trigger point codes and then used in macros or algorithms to 
determine Interactive audio and/or graphics responses. 

The presentation of the graphics interrogatoiy messages can also be 
made a function of subscriber selections to previous interrogatoiy 
messages. The logic used in the codes for selecting the next graphics 

10 message is similar to that used for selecting audio messages. One 
method, as disclosed in earlier ACTV patents, is the "decision tree" logic 
methodology. The subscriber makes a selection to a first predetermmed 
interrogatory graphics message. After the subscriber hears an 
appropriately branched audio channel, the processor (178) wfU. interpret 

15 ^phics algorithmic codes sent down ftcm the operations center (608) 
and 'Will read from memory ^84) an iq^ropiiate next graphics message. 
The processor (178) then directs the character generator (624) to 
overlay the selected graphics message onto the next frames of video. 

The voice/data extractor (174) of the preferred embodiment shown 

20 in figure 3 is shown in more detail in figure 4. This particular extractor 
corresponds to the Pulse Amplitude Modulatton (RAM) inserter (219) 
discussed betow in connection vAih Figure 9. The audio and data is 
stripped out of the baseband signal in the four-channel VBI extractor 
(174). The signal separator (174) essentially reverses the process by 

25 ^ch the audio and data signals were inserted into the video signal. A 
video digitizing system and timing controller (320) digitize the incoming 
video signal at the same sampling rate at vrbich the PAM audio samples 
are inserted into the video signaL Alternatively. If a digital adaptive pulse 
code modulation (ADPCM) extractor, corresponding to an ADPCM 

SO inserter described below, is used in the interactive system, the extractor 
does not include the video dIgltlTing and bufiier drcuitiy shown in Hgure 
4. 

The timing controller (320), shown in Figure 4, determines which 
lines of video contain the audio signals and separates the compressed 
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audio signals by directing each audio cfaahnel Into an associated FIFO pair 
(328-356). As wtXh the InsertloA circuit the FIFO's in the FIFO pairs 
(328-356) are used alternately so that xvfaile one FIFO is reading the audio 
data, the other is mltlng data. Tlie digital-to-analog (D/A) converter 
5 (360) sequentially reads from the FIFO for each channel which is in the 
wiite mode and converts the digital signal for that channel into an audio 
signal. The analog ou^ut for each channel is filtered by filters (364-376) 
and directed to buffers (380-392). The buffers (380-392) preferably 
acQust and amplify the analog output signals to -IV to -i-lV range. The 

10 four output audio signals connespond with the four audio Input channels. 

Aa shown in Figure 4. the line 21 data is read into 8-bit shift 
registers (400) vAierein the 16 bits of serial data are converted to two 
bytes of data. The line 22 data is read by 8-bit shift registers (402) and 
the 16 bits of serial data are also converted to two bytes of data. 

15 Hie Une 21 and 22 data (i.e.. the data codes) are directed firom the 

extractor (174) into the microprocessor (178). The microprocessor 
(178) decodes the data and stores it in either RAM (282) or ROM (286). 

In an alternative voice/data digital extractor embodiment at the 
receive end. the composite digital signal is received where the FEC code 

20 is applied fay an FEC decoder (516). as shown in Figure 5. The resulting 
signal is applied to a digital demultiplexer (520) where the audio, data 
and video components are separated and individually buffered in a set of 
buffers (524-532). From buffer A (524). the composite digital audio 
signal enters a digital-to-analog converter (536) where the plurality of 

25 audio signals (Including the standard audio channel and the additional 
audio channels) are separated, decoded and decompressed to recreate 
each Individual analog audio channel. FVom buffer B. the digital video 
signal similarly enters a digital-to-analog converter (540) where the 
signal is decompressed and decoded to form an analog video signal. Hie 

30 data is simllaily decompressed and decoded In analog-to digital decoder 
0(544). 

Refemng back to figure 3, the processor (178) receives data from 
the extractor (174), stores data and reads instructions from memoiy such 
as temporary RAM (282) memoiy, ROM or FLASH ROM (286) memoiy. 
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and receives response data from the remotes (604) via an infrared 
interfkce (262). or alternatively a computer keyboard. Optionally, the 
processor (178) can interface to a printer (302) via a printer port (302) 
for printing out coupon codes, as discussed below. The processor (178) 
5 is preferably a 80C188 based microcontroller, but can be any similar 
controller. Aa mentioned above, this entire system could be built into a 
personal computer using a standard off the shelf multimedia personal 
computer (187) v;ith preferably an 80386 or hi^ier based processor. 

The operating system software for the processor (178) is preferably 

10 stored in FLASH ROM memory (286). The FLASH memory (286), as 
shown in Figure 4. also prefeiabfy stores new instructions to use with 
reprogrammable software. It works in conjtmction with RAM (282). The 
RAM (282) can also be used as an Intetfiace to a print buffer. The print 
buffer is used so that data transmissions to ttie site can be buffered, error 

15 corrected and formatted for printing. The site controller (178) is 
preferably coimected to a printer via a parallel printer port. The 
controller (178) sends cottons to the printer (302). 

The interactive program box (600) connects with the remotes 
(604) preferably by infrared (IR) connection. The processor (178) 

20 preferabty^ stores the setup commands and interactive branching 
commands and algorithms in associated memory, as described below. In 
the present invention, code interpretation and switching is preferably 
accomplished by the interactive program box processor (178). The 
processor (178) collects subscriber responses from the remotes (604). or 

25 computer keypad. 

The advantages discussed above in relation to presenting an 
interactive program using trigger points are obtainable in each of the 
interactive program box embodiments shown in figures 6-8. In the 
embodiment shown in figure 6. altemattve audio segments are preferably 

30 sent serialfy from the opoations center (608) in the SAP channel. The 
demodulator (617) receives a composite interactive signal comprising the 
standard video and standard audio signal along with an audio subcarrier. 
The demodulator (617) breaks the signal into it's component parts, 
forwarding the baseband video to a data extractor (175) and the standard 
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audio to an audio switch (620). Hie line 21 data extractor (175) takes 
out the data codes. Including the trigger points. 

The SAP channel comprises a pluraUty of audio segments lined up 
serlaUy. The audio sjegments are digitized In the analog to digital 
converter (175) and are preferably stored in digital audio memory (283). 
At certain times during the program, data codes will designate a trigger 
point and key the microprocessor (178) to select and play an audio 
segment corresponding to previous user Input(s). according to the 
process described above. The microprocessor (178) calls the appropriate 
audio segment(s) from Internal memory or external data storage (629) 
and commands the audio switch to pass the selected audio segment to 
the RF modulator (622) for play to the subscriber. At the end of the 
InteracUve time period, the controller (178) Instructs the audio switch 
(620) to again pick up the standaid audio. 

In an alternative embodiment similar to that as shown In Figure 6 
and discussed above, the simple addition of a second tuner, receiving the 
composite RF signal, could be used to tune to a second audio channel for 
collection of t r a n smi t ted audio segments. The tuner would pass the audio 
segments to the A/D converter (175) with the operation of the rest of the 
interactive program box (600) similar to that described above in 
connection with Figure 6. 

Figure 7 shows another Interactive program box (600) embodiment 
for providing alternative audio and graphics segments. This embodiment 
uses two tuners: an RF demodulator (616) and a data tuner (615). The 
RF demodulator (616) tunes to and demodulates the conventional video 
and audio signal In the standard video bandwidth. The data tuner (615) 
receives a single digital audio sl^iaL The signal comprises digital serial 
audio segments modulated onto an analog carrier. The data tuner (615) 
demodulates the signal Into digital audio. The digital mter&ce selector 
and error corrector (177) separates the audio segments and performs 
error correction according to any error correction scheme commonly 
understood In the art The controller (178) directs the selector (177) to 
extract selected digital audio segments from the serial digital stream and 
send them to the digital audio memory (283). Selection of one or more 
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audio segments for pli^ as personalized messages on the television set 
(186) occurs according to the processes described above. After the 
controller commands the memozy (283) to forward a digital audio 
segment, the segment Is converted to analog hy the digital to audio 
5 converter (176) and Is subsequently passed to the RF modulator (622) for 
play on the television set (186). 

Another Interactive program box (600) embodiment for receiving, 
storing and selecting alternative audio segments is shown In figure 8. At 
the operations center (608). the audio segments are digitized, time 
10 division multiplexed, modulated and converted to frequencies in unused 
channel frequency space in the cable television spectrum. e.g.. cable 
guard bands. 

Tlie RF donodulator (616) again demodulates the conventional 
video and audio signal. The data exbactor (175) receives the signal and 

15 extracts the VBI line 21 data codes. The data In the VBI indicates the 
frequency ch a nnel s in v^ch the digital audio s^ments are transmitted. 
For example, audio messages A-E are located in between ohn««^ift 14 and 
15. The controller (178) instructs the data tuner (615) to tune to that 
part of the spectrum between channels 14 and 15. Alternatively, an 

20 autotune capability can be used to find the audio channels in the 
spectrum. 

Tlie tuner (615) demodtdates the digital audio signal and forwards 
the signal to the digital demultiplexer (700). The demultiplexer (700) 
demultiplexes the signal Into n digital audio c^'flnnfls and forwards each 

25 channel to a separate D/A converter (702-710) vrbert each of the digital 
channels are converted to analog audio. As described above, one of these 
chazmels can be selected as identified at the trigger points for play over 
the audio speaker to the subscriber. All of the other components in 
Figure 8 operate identically to that previously described in connection 

30 with figure 3 to provide Interactive or conventional programming to the 
subscriber in the home. 

The embodiments described above and shown in connection with 
figures 3-8 relate to different ways of receiving broadcast audio segments. 
Alternative^, interactive audio segments, or graphics elements, could be 
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prcstored on cartridge. CD ROM or even floppy disk. However, the 
trigger points are preferably broadcast as data codes embedded in the 
conventional video signal and will work with any of the embodiments 
described above. 

In this manner, each subscriber at different homes hears a dlflferent 
and personalized response from their own television set (186) due to 
their interaction with the Interactive program box (600). As discussed 
below, the power of this system can be Increased exponentially via 
preloading of additional audio segments expanding personalized delivery 
and the mec h a ni s m of memory branching herein the system actually 
remembers and uses the subscriber's earlier selections to present or 
compose unique messages to the subscriber. 

The range of possible audio responses can be dramatically increased 
by preloading audio segments. First, more than simply the audio 
response segments broadcast along with a trigger point can be 
transmitted by preloading the memory at the interactive program box 
(600). Preloading can be accomplished by sending additional audio 
segments in a time period prior to that period in which the normal 
broadcast audio responses are sent. 

For example, at the begizuilng of a program several audio segments 
can be broadcast to the interactive program boxes (600) by any of the 
methods and systems described in coimectlon with figures 3-8. The 
controller (178) directs that these segments be stored in memory. Each 
audio segment has a pointer which identifies the particular interrogatory 
message corresponding to an audio segment. For example, if the 
interrogatory message presented early in the program was the following, 
•Are you a boy or a glri," an audio segment vrbldii might correspond to 
this question is "Oh. I see. you are a boy." At the time of the interactive 
element, identified to the controller by the trigger point codes, 
preferably more audio responses associated with the earlier presented 
Interrogatory message are output from the operaUons center (608). 
passed through an audio mixer (118). embedded in the video signal at the 
Inserter (130) and transmitted to the remote sites. Once received at the 
Interactive program box (600), the audio responses are extracted from 
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the video signal and forwarded to the switch (620). The controller (178) 
then either selects an audio response made of prestored audio segments 
in memory or one of the audio segments broadcast with the trigger point 
codes. The switch (620) branches to the appropriate channel to forward 
5 either one of the segments firom memory or (me of the recentty broadcast 
segments. At the end of the interactive period, the switch (620) 
branches back to the standard audio. 

Even more enhanced and flexible operation can occur through the 
addition of fxtfr"^i data storage (629). such as CD ROM or cartridge. For 

10 example, sports statistics or other information on athletes or others can 
be stored in CD ROM. During live sports event either audio segments or 
graphics displays focusing on the athlete can be called by the processor 
(178) and presented to the viewer as a function of user sdection of an 
option or at a trigger point if the user indicated during queries at the 

15 beginning of the live event that th^ were interested in a particular 
player. 

In the present invention, the interactive audio branching is 
preferably performed in the interactive program box (600). The 
branching algorithm, preferably defined by a series of branch codes, can 

20 be Inserted as part of the program. The processor (178) stores the 
algorithm in associated RAM memoiy (282). The algorithm determines 
ttie proper channel to switch to as a function of the subscriber selection. 

The branching algorithm can be expanded by utilizing previous user 
selections in making a decision on a current audio segment This is the 

25 memory branching embodiment. In other words, previous user 
responses, stored in RAM memoiy (282) in the interactive program box 
(600). are used as input to a logic network, or algorithm. Successive 
audio output segments can be related in a "decision-tree" type 
relationship. The logic network, or algorithm, processes the data input 

30 for selection of an audio output response message. This memory 
embodiment goes well with the eiq)anded audio response capability 
described above. 
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The memory branching technique, described above, can work 
similarly for basing successive graphics interrogatory messages on 
previous subscriber selections to earlier graphics Interrogatoiy messages. 

Another embodiment uses -stacking- to Increase the audio answer 
response opUons. The expansion is due to Unking separate audio 
segments separated in time to form a greater number of answer 
responses. Hie foUowlng example Iflustrates the use of stacking as wcU 
as memory branching, described above. The left hand column of the 
example shows the Interactions between the program and the subscriber. 
Tbe right hand column shows the logic applied and branching peribimed 
in the Interactive program box (600). 



1. Can you ten time? 

(1) YES 

(2) 

(3) 

(4) NO 



RETURN FROM BRANCH 



AcrrowTOTw 

• Store button pressed In memoiy A 

• ffbuttonsl.gototzack 1 

• tfbuttOD=2,gototxackS 

• ffbuttonsS.gototrackS 

• irtnxttQas4«gototxack2 

• tfno button pressed, go to trade 3 

GO TO TRACK 1 



2« What time Is it? 

(1) 2:30 

(2) 3:30 

(3) 3£0 
(4} 4 A) 



• Store button pressed In memotyB 

• ffbuttooslfgototxack 1 

• irbuttaQs2,gototxack2 

• irbuttans3«gototzack3 

• ffbuttons4«gototzadc4 

NOIE: 3:30 (button (2)) is the contct answer! '^'^^P^SP^ot^A 



Recall response fiom question 1 
TRACK 1: 'You said you knov7 
honrtotelltlme../ 
1RACK2: Tou said you dont 
know how to tdl HTn#> . ^ / 
TRACK3: Tou didnt Indicate If 
you can ten time. 

Memoiy branch 

TRACK 1: •••.andyouwerent 

kidding. Great Workl* 

TRACK2: but you sure learned 

fast That's greaf 

TRACK3: and with more work, 

you will learn more/ 

TRACK 4: but with more work, 

you win leam more/ 

RETURN FROM BRANCH 



• Recan memory A and branch as fonows: 

• umemoryA^rl.gototrackl 

• IfmemoryAB4,gototiack2 

• irmemoryA;«2or3,gototzack3 

• Branch as follows: 

• ff memory Agl amj ynCT Miy B=2 
(correct) go to track 1 

• If memoxyA^l and memory Bs2 

(correct) go to track 2 

• IfniemoiyAsl andmemoryB=l, 3 
or 4 (Incorrect) go to track 3 

• memory A^bi and memory Bslt 3 
or 4 (incorrect) go to track 4 

GO TO TRACK 1 
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Stacking is evident In the above example by the increased number 
of optional audio messages formed by linking the later two branches 
indicated In the example above. The possible branch combinations are 
provided* as follows: 

Branch #1 to Branch #1 

Branch #2 to Branch #2 

Branch #3 to Branch #2 

Branch «1 to Branch #3 

Branch #2 to Branch #4 

Branch #3 to Branch #4 

Tberefore, the present invention covers numerous methods for 
Increasing the interactive audio response capabilities. 

C Subscrlbw Tnt>>rfaP#> 

Each subscriber interacts with the system via a subscriber Interface 
(or "remote") (604) or a computer keyboard. 

Ihe keypad (604) preferably comprises a number of functional 
keys. The keypad (604) preferably comprises k^ labelled 0-10. "YES" 
and "NO" labeUed keys, and "TRUE" and "FALSE" labeUed keys can be 
accommodated hy the present invention. The invention can also 
accommodate an LCD panel for receiving messages and confirming 
answers. Similar functions can be assigned to computer keyboard kejrs. 

Addressabillly and authorization occur preferably using techniques 
wen known in the cable industry. For example, program selection and 
authorization can occur using menus on the screen with cursor overlays. 
Tlirougb the menu ^tem. a viewer can access a program by inserting a 
unique ID code, bUIing number, etc. Macro access algorithms stored in 
memory In the Interactive program box enable entry. 

Either at the begirmlng of the Interactive program, as in the 
preferred embodiment, or alternative^ as the program progresses, the 
subscribers are requested to enter responses via the keypad (604) or 
computer keyboard. These requests are preferably displayed on the TV 
monitor (186) or computer screen through the use of graphics overlays. 
The interactive programming preferably consists of graphics 
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interrogatory messages ^th correct and incorrect answers overlaid on 
the video. 

The keypad (604) or computer keyboard allows the subscriber to 
interface with the interactive programming by depressing keys 
5 corresponding to answers of an interrogatory message. For example, in a 
multiple-choice question mode, each participant selects a keypad button 
corresponding to his or her choice, lliese selections are preferably 
stored and later used by the controller (178) in selecting either an audio 
or graphics message and either directing the audio switch (620) to 

10 switch firom the standard audio channel to an alternative audio channel 
for passing the respective audio segments to the television (186) or to 
direct the graphics character generator (624) to overlay graphics 
characters over predetermined portions of the screen. The user 
selection is mapped into one or the pliirality of audio responses. 

15 comprising one or a combination of linked audio segments, or one of a 
plurality of graphics messages after the user selection is applied to a 
some functional algorithm. 

IX Program Prcparatton 

For the present Invention, it is anticipated that the majority of the 
20 interactive programs will be produced ofif-line and thereafter recorded 
for subsequent broadcast on some storage meditun at a central location. 
Alternatively, live programs could be broadcast with the interactive 
elements inserted in similar &shion to those techniques described in the 
parent case. 

25 The broadcast or operations center (608) is where the task of 

either recording and broadcasting or creating and packaging the 
composite interactive programs is preferably performed. These 
programs are created to be received and viewed as both regular 
conventional television programs and as an interactive programs. 

30 TTierefore. they must be created in such a manner that there is no 
noticeable difference to those noninteractive subscribers watching the 
conventional program when the interactive portions begin for authorized 
interactive subscribers. Likewise, when the interactive portions end and 
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both groups of viewers again view the same video and hear the standard 
audio, there should be no noticeable transition. 

The composite interactive programs that can be created include 
educational programs, home shopping programis. movies, children's 
5 programs, news programs, etc. The stored programs may be in analog or 
digital form and stored in permanent or volatile inemoxy sources. In 
addition, live programs can be enhanced with interactive elements and 
broadcast to subscriber homes as part of the present invaition. 

The interactive program used in the prosent invention preferably 

10 comprises a single video, a standard audio channel, graphics and data 
codes, and a plixrality of audio signals embedded therein to achieve 
interacttvUy. However, as described and shown above in coimectton with 
figures 3-8, the audio can be sent on separate channels, preloaded at the 
interactive program box (600), or stored separately in external data 

15 storage. 

Any of these embodiments will work with the interactive trigger 
points of tiie current invention. The program must first be produced in a 
studio. The resulting program will comprise the conventional video and 
audio receivable on any television set (186). Next, the desired interactive 

20 sequences must be defined. The producer must decide which portions of 
the program will be interactive. Tlxese portions will be identified by 
trigger points. These trigger points correspond to the times wbexi the 
program can be altered. The trigger points are essentially codes that will 
call macros that will be used to trigger the occurrence of eadi interactive 

25 sequence during the program. 

Therefore, the producer must decide at what points in the program 
to include the interactive responses, i.e. the trigger points. The 
interrogatory messages required for soUdting viewer interests and 
answers must also be scripted. The alternative audio and graphics 

30 responses to possible viewer selections are also created by the producers. 
Next, the interrogatory messages, with possible ^tem responses, must 
be mapped into predetermined trigger points. This mapping is made 
possible by developing an algorithm on a personal computer. The 
algorithm maps specific system responses (audio or graphic) to occur at 
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the trigger points as a function of viewer selectlon(s). For example, if the 
user selects A to interrogatory #2. a predetermined audio or graphics 
re^onse Is selected to play at an Identified trigger point 

Next, software in a piersonal computer coixq)lles the above mapping 
5 Into program transmission codes. The personal computer then controls 
the loading of the audio and data Into the program stream at the 
appropriate times and Inserts the codes Into fields. Timing and control 
between the video and multiple audio elements, discussed above, are 
provided through the use of data codes. The data codes are stored In 

10 memoiy in a personal computer as part of the ACTV programming 
language. The codes comprise commands, or branch codes, for 
branching between channels (as discussed in the aforementioned ACTV 
patents and applications), timing signals for controlling the interactive 
program, codes defining eadi trigger point, data or text, commands for 

15 termination and initiation of interactive programming viewing, or triggers 
for executing macros. 

Flnall}r. the operating system software must be developed and 
loaded into the interactive program box (600). This software can be 
uploaded to the box on the cable and stored in RAM memozy (282). 
20 loaded onto a cartridge or placed in ROM memoiy (286). Tills software 
comprises the macros that interpret subscriber inputs and map these 
inputs into specific interactive elements at a trigger point during the 
program. 

At this point the audio segments can be stored in a mtiltlchannel 
25 tape storage device or digital device for later use. When it is time for 
broadcast the video and standard audio pass to an inserter (130) to be 
combined with the data codes and multiple audio rimni^fiff at the 
appropriate time as commanded fay a processor. As shown in FIguze 1. 
the audio channels are preferably fed from the personal computer (114) 
30 into an audio mixer (118). Furthermore, this processing can be utilized 
to align the dlfiierent audio response tracks in time. 

The analog inserter (214) of a preferred embodiment is shown in 
more detail in Figure 9. The four audio ou^t channels are fed into the 
inserter (214) to be mixed with ttie video and data. Each audio signal is 
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preferably an analog signal. A buffer (222) Is provided for each audio 
signal to electrically isolate the audio inserter (214) from the source of 
the audio signals. An amplifier amplifies the signals in order to converts 
the analog audio signals to a format which is fully compatible with the 
analog-to-dlgltal (A/D) converter (234) discussed below. 

The output of each buffer (222) is directed to a bank of low pass 
filters (226), one for each audio signal. In addition to reducing high 
frequency noise fimm the audio signal, the low pass filter (226) dips the 
audio signal to a desired fi«quency. The filtered audio output signals of 
the low pass filters (226) are directed as inputs to a multiplexer (230). 

In a preferred embodiment the multLplexer (230) is controlled by 
timing signals received from the timing circuit (254). The timing circuit 
(254) is synchronized with the video signal of the interactive program 
and controls the multiplexer (230) to sequentially take a sample from 
eadi audio signal and to direct the sample to an analog-to-dlgital (A/D) 
converter (234). The A/D converter (234) converts the samples of the 
four audio signals to an 8-bit digital format 

As shown In Figure 9, each of the four audio channels preferably has 
an associated first-in first-out (FIFO) pair (238. 242). The 8-bit digital 
samples for eadb. signal are directed as an input to one of the FIFO pair 
^38, 242) associated with that signal. The samples for eadi channel are 
preferably directed to one of the FIFO pair (238, 242) for that chaxmel 
until enough samples are in that FIFO to fiU the desired number of lines 
of the video field into which the sampled audio is to be inserted. Tlie 
audio samples are then directed to the other FIFO of the FIFO pair (238. 
242) for that channel until that FIFO has the correct number of samples. 
In the meantime, the contents of the first FIFO are written onto the video 
slgnaL It is foreseen that other types of memory means may replace the 
FIFO pair (238. 242). 

When the content of a particular FIFO is to be inserted into the 
video, the digitized audio samples stored in that FIFO are directed (first- 
in-first-out) to a digital-to-analog (D/A) converter (246). as shown in 
Figure 9. Ihe D/A converter (246) converts the stream of digitized audio 
samples into an analog PAM (pulse amplitude modulated) signal. A 



32 



wo 96/05699 



PCTAJS95/04620 



second buffer (250) is provided for electrlcaUy Isolating the inserter 
circuit from the switching circuitry (258) which multiplexes the audio 
s ignal s into the video signal. 

In a preferred embodiment the audio signals A and B are each 
inserted independently onto 5 lines of one field of the video signal, and 
audio signals C and D are Inserted independently onto 5 lines of the other 
Add of the video signal. In order to enable continuous audio playback for 
each channel, the audio for each channd must be compressed so that 5 
lines of video per frame contain enough audio content for the playback 
time of one frame of video (about l/SOtfa of a second). 

The number of samples of audio required for each channel depends 
upon the desired sound quality. If lower sound quality is desired, fewer 
lines of video may be used, and vice versa. 

In the preferred embodiment five (5) lines of video are used per 
frame per channel of audio, with 105 samples per line at a sampling rate 
of 15.734 samples per second. The multiplexer (230) sequentlaUy 
samples each audio signal 15.734 times per second and altemaUvely 
dfrects 525 samples to each of the FIFO's (238. 242) associated with 
each audio c h a nnel . At a predetermined time before the audio from each 
channel is to be Inserted Into the video signal, the timing circuit (254). 
shown in Figure 9. instructs the fuU FIFO for the channel to write the 
digital audio to the D/A converter (246) wtdch converts it to a PAM 
signal. The timing circuit (254) then switches the video selector (258) 
to receive the PAM audio signal throu^ the buffer (250) as an input in 
Ueu of the video signaL In this manner, the additional audio rhannels are 
inserted into the video. The standard audio remains unaffected in the 
normal audio canler. Once the audio has been inserted Into the video at 
the desired location, the timing circuit (254) commands the video select 
switch (258) to change its input back to the video signal. Hiis process is 
repeated for each audio signal at the video lines ix^erein that audio signal 
is to be Inserted into the video signal. 

T^ically. ten lines of the Vertical Blanking Interval (VBI) of the 
video signal are available for use in each field of video. In the preferred 
embodiment, for each frame of video, audio signals A and B are each 
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inserted on five lines of the VBI of one field, and audio signals C and D are 
each inserted on five lines of the VBI of the other field. The desired 
audio quality and number of audio channels dictates the number of lines 
of video vrbiich must be used to transmit the signals. Preferably, all of the 
5 additional audio channels can be inserted into the VBI. However, because 
of regulations regarding broadcast transmission^ and other uses of the 
video lines, if more lines are required for the audio services then active 
lecture lines may have to be used. These active lines can be grouped and 
placed at the top or bottom of the picture which effectively will nor 

10 interfere with the viewed video due to the overscanning of most 
television sets. Alternatively, the lines can be randomly interspersed 
throughout the program. 

In the preferred embodiment, data Is preferably sent on line 21 of 
the vertical blanking interval (VBI) using the predefined standard for 

15 transmitting data on line 21. The line 21 data Is preferably sent as 16 
bits of serial data per field, and is placed Into the video signal using a 
standard line 21 encoder. However, the data can alternatively be inserted 
on other lines of the video. 

In an alternative embodiment instead of inserting analog RAM 

20 samples as shown in Figure 9. midti-level digital samples, such as ADFCM 
samples may be inserted into the assigned VBI lines. In this 
embodiment, a bandpass filter would filter the wideband analog audio in 
the desired frequency range. The signal is then sampled at some 
firequency rate In a sampler. Next the RAM audio samples are quantized 

25 into one of n levels. An encoder then maps the amplitude levels selected 
by the quantizer into preferably a three bit ADPCM samples. These 
samples are buffered and preferabty converted from binary to an 8-Ievel 
converter. The ADPCM audio samples are then Inserted in the video at 
the direction of a processor into the designated VBI lines. At the 

30 receiver, the corresponding extractor (174) sends the signal to a digital 
to analog converter and subsequent^ to a low pass filter to receive the 
filtered analog ou^ut This embodiment has the advantage of reducing 
end-to-end quantization noise and provides some degree of immunity to 
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channel noise and to noise resixltlng from VCR recording and playback of 
the video/analog signal. 

Several additional audio signals may be transmitted along with the 
Interactive video signal in other ways. An audio signal may be transmitted 
over each of the normal audio channels. The audio may be distributed in 
many ways using the available audio channels. For example, high quality 
background music for the Interactive program could be transmitted over 
the hoimal audio c h a nn el. Then, lower quality audio for voice-overs could 
be inserted onto the video signal and/or using the SAP signal. Preferably, 
the audio segments are aligned serially in the SAP channel. This would 
enable a larger number of audio signals to be transmitted with the 
Interactive program. In addition, the audio channels could also simply be 
combined with the video and data signals via a frequency division 
multiplexer. The audio channels can be digitized using any common 
analog-to-digital conversion technique such as PCM, DM. ADPCM, etc. 
Finally, vocoders (e.g.. LPC. APC. channel vocoders) can be employed to 
reduce the required data rates and bandwidth requirements for 
transmitting voice. 

Tlius, there are many variations In wbich the present invention may 
be used to facilitate the transmission of multiple audio channels. In one 
possible alternative embodiment. Instead of embedding the plurality of 
audio channds In the video signal, the channels are all sent as one audio 
signal using serial sequencing. Serial sequencing is defined by reading 
the audio tracks from memory serially In sequence (I.e.. Al. then A2, then 
A3, etc.). The resulting signal Is then sent as a secondaiy channel, or SAP 
channel, with the primaiy channel containing the standard audio. In this 
embodiment these particular audio tracks would have to be sent prior to 
the time when the answers are scheduled to be played to the subscribers. 
Therefore, when the SAP channel is received at the interactive program 
box (600), the signals will be Independently decoded from the SAP 
channel, buffered and stored in memory (284) for later use when it is 
appropriate to playback the responses to the subscriber. Tlie interactive 
program box (600) for this embodiment Is described sibove and shown in 
figure 6. 
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Alternatively, the interactive system of the present invention can 
provide for transmission of the interactive signal by digital transmission 
means, resulting in reduced bandwidth requirements. In the digital 
transmission embodiment the interactive signal can be transmitted by not 
5 only sateUite but ATM, Sonet. Tl or any other digital transmission 
system. 

The digital transmission components at the operations center 
(608) are shown in Figure 10. As shown in Figure 10, the plurality of 
audio signals are ouQmt firom a personal computer (114) or tape storage 

10 device and mixed in the audio mixer (118) with the conventional audio, 
preferably resulting in four ou^iut channels. The audio signals are then 
preferably sampled, encoded and compressed In the audio digital 
^coder/compressor (500). The encoding technique can be a waveform 
coding technique such as PCM, ADPCM or DM. Alternatively, the signals 

15 can be encoded using synthesizer or vocoder techniques such as 
MUSICAM. Linear Predictive Coding (LPC), Adaptive Predictive Coding 
(AFC), and Sub-band coding. A single composite audio signal is output 
firom the audio digital encoder/compressor (500). 

As shown in Figure 10, the video signal is input into the video 

20 switch (108) which outputs one of the video ^gnals. The video digital 
encoder/compressor (504) takes the video signal, encodes and 
compresses the signal according to a known standard such as MPEGr-1. 
MFEG-2. JPEG, or other DCT based coding scheme, wavelets, firactals or 
other transform based approaches. Altonatlvely, the video digital 

25 encoder/conq)ressor (504). shown In Figure 10. could utilize a waveform 
based technique, such as PCM, ADPCM. or DM, for encoding and 
compressing the video signaL 

The data firom the personal computer (114). in Figure 10. is 
preferably compressed via a board in the computer (114) and ddivered 

30 directly to the digital multiplexer (508). The digitized data, video and 
audio signals are delivered to the digital multiplexer (508) where the 
signals are buffered and combined via time division multiplexing, with 
appropriate synchronization signals applied, to form a composite sync 
serial data stream. At the output of the digital multiplexer (508). forward 
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error correction fFEC). the code depending on the selected transmission 
path, is appUed by an FEC encoder (512). The composite Interactive 
digital signal is then transmitted to the receivers (158) via a suitable 
traxismlsslon means such as satellite, fiber optic, cable. DS. etc. 

In summary, the operations center (608) creates and transmits an 
interactive video signal by taking video, inserting multiple audio channels 
and data codes, and transmitting the resulting interactive video signal to 
the transmission system. 

£. The TransnHssInn SvB»Ar^ 

As shown in Hgtire 1. the system uses an interactive program 
deliveiy system (98) vrtth any broadcast transmission means including 
satellite (150). cable (138) or television (142) to deliver the interactive 
program for distribution to the subscriber homes. Likewise, distribution 
of the signal at the receive end can be by any suitable tiiansmlsslon means 
(166) Including a cable distribution system, fiber optic, etc. In a satellite 
broadcast transmission (150). the Interactive video signal Is transmitted 
to uplink equipment (146) where it may be multiplexed, upconverted. 
modulated, amplified and transmitted by satellite (150) to a receiver site 
(154) for distribution to the classroom locations. In the analog 
environment the interactive video signal enters an analog receiver (158) 
and then preferably a cable distribution system (166) that directs the 
signal to subscriber locations. In the alternative digital embodiment, the 
composite digital interactive signal enters a receiver (158) where it is 
demodulated and then preferably passed to a digital cable distribution 
system that directs the signal to the subscriber locations. Although a 
cable distribution system (166) is the preferred transmission media to 
the remote sites, the Interactive video signal may also be distributed by 
any conventionally known technique, such as DBS. fiberoptics. low or 
high power broadcast television, telephone lines, cellular networks, and 
similar technology can be used Interchangeably with this program 
delivery system. 

P. Return Path 

If desired, status messages and participant Infoimation can be sent 
back from the remote sites to the operations center (608) via return 
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paths. For example, subscriber selections could be stored in RAM (282) 
and later forwarded to the operations center (608) to aid in creating a 
Subscriber profiles for taxgeted advertising. The return paths can be dial- 
up tdephone lines or via an X.25 packet switched network, cable. 
5 wireless networks, etc. The preferred method of sending this 
information back is based on subscriber initiation. The alternative 
method of sending this Information is by polling the remote sites from 
the operations center computer (114). Alternatively, the remote sites 
can send perfonnance information and messages after consolidation, at 
10 pexlodic Intervals. 

If the interactive program box (600) functions are performed at the 
cable headend as part of a multi-user system, as described above, then the 
return path takes on greater importance. However, the path to and from 
the cable headend can be viewed slmpty as an extension of the signal path 
15 between the interactive program box (600) in the home and the 
television set (186). 

hiput can be transmitted back to the operations center (608) in a 
two vray caUe system or in a tdephone eystem via a modem (312). The 
modem (312) is connected to the controller as shown in figures 3 and 6- 
20 8. The controller could send to the modem (312) a series of data 
streams characterizing the viewer's particular interests. 

III. System Qp>rnHftn 

The simulcast system operates through the preparation, 
transmission and reception of a composite interactive program that can 

25 provide an interactive program to those subscribers* homes equipped 
with an Interactive program box (600) and a regular television program 
for those homes not so equipped. As mentioned above, this dual 
functionality Is provided by preferably embedding in primarily the 
nonvisual intervals of a standard video signal a plurality of audio segments, 

30 graphics data and programming control data, the reception and 
processing of which provides a fully interactive e3q)erience for 
subscribers. Altematlvety. the audio segments can be prestored at the 
interactive program box (600). stored in external data storage (629). sent 
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down in serial fashion on the SAP or sent serially or In parallel on a 
separate channel. 

Information Is stored on each audio segment* Each audio segment 
comprises messages In response to user selectlon(s) on the keypad (604). 
5 When the program begins, an INITIATION command is sent from the 
broadcast or operations center (608) to all the interactive program boxes 
(600). The television set (186) is directed to default to a predetermined 
channel (e.g.t channel 1). 

Initially, the unauthorized and authorized interactive subscribers 

10 view the same standard television program with both viewing the 
standard video and hearing the standard audio. For noninteractlve 
subscribers, the standard video and audio is all they view and hear for the 
entire program* The interactive subscribers, on the other hand, are able 
to interact with the program during designated time periods. 

15 At some point in time, trigger point codes are embedded in the 

composite interactive program signal, the trigger points indicating the 
initiation of an interactive program. The VBI data codes Immediately 
inform the interactive program box (600) that the program is interactive. 
The controller (178) begins decoding the codes after detection. 

20 Graphics codes will also be sent as part of the signal. These codes enable 
the interactive program box processor (178) to read the graphics bit 
maps from memozy (282, 286), corresponding to the codes, and direct 
the character generator (624) to overlay the graphics onto a 
predetermined portion of the screen. In this manner, the first graphics 

25 interrogatoiy message can be displayed on the screen. 

While In the prefexred embodiment and for purposes of illustration, 
the graphics messages are interrogatory, they may also be instructional, 
request user to select amongst several options to receive special features, 
provide status, or request subscriber profile information from the 

30 subscriber. Tlie graphics overlay may consist of simple alphanimieric 
characters forming a message or more sophisticated characters such as 
pictorial cartoon characters. While both the interactive and 
noninteractive subscribers are hearing the standard audio, the Interactive 
viewers are requested through the use of the graphics overlay to enter a 
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choice on their keypads (604) or if the viewer is interacting via a 
personal computer (187). the keyboard connected to the personal 
computer (187). 

In this manner, at the next trigger point, the controller directs the 
audio switch (620) to cut away from the standard audio and begin to play 
an introductory message such as 'Hi, welcome to the show, this is an 
interactive show. Would you like to interact with the show?" If the 
viewer desires to interact, he or she depresses the YES button on the 
remote, and tbe interactive show will begin. If the viewer does not wish 
to interact with the program, the controller directs the audio switch to 
branch back to the standard audio and the television will pick back up the 
conventional television program in progress. 

Simultaneously, a BEGIN INPUT command is sent to all of the 
interactive program box controllers (178) instructing then to accept 
input from the infirared remotes (604). The last input before the 
expiration of the INPUT period is accq>ted as the subscriber's selection 
fay the system. 

After the viewer has property indicated their desire to interact with 
the program, preferabfy a set of graphics interrogatoxy messages wUl be 
presented one after another to solicit user selection useful for obtaining 
desired viewer information. For example, assume the show being 
presented is a show on redecorating a kitchen. First, an audio segment 
will be played stating *H1, this is a show on redecorating your kitchen, 
are you interested in redecorating your kitchen. YES or NO"? If the user 
selects the YES button on the remote, anotiier audio segment may play 
"Well, thank you very much. Let me ask you a couple of quick questions 
tm the displiQr.* At this point, the micnqmcessor (178) accesses a series 
of graphics interrogatory messages stored In memory (282, 286) and 
dfrects the overlay chip (624) to display the graphics messages in 
predetermined order. The user responses to these questions are used 
later in the program at particular trigger points for presenting interactive 
messages to the viewer. A series of sample interrogatory messages may 
be displayed In succession as follows: 

"DO YOU HAVE WINDOWS IN YOUR KTTCHEN? 
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A YES 
B NO.- 

•HOW BIG IS YOUR KITCHEN?* 
A. 10»20 B. 20*30 C. 30*40 D. 40*50 E. 50*60" 
5 "WHAT IS YOUR COLOR SCHEME? 

A WHTTE B, BROWN C. BLUE D. GREEN E. YELLOW 
"WHAT T5fPE OF FLOOR DO YOU HAVE? . 
A WOOD B. FORMICA C. TILE" 

Preferably, no audio accompanies the display of the graphics 
intcrrogatoiy messages. The selecUons made by the user above are 
received by the interactive program box (600). tagged with a code 
designating the particular related question and stored In memory (282 
286) for later use at the appropriate time. When a trigger point is 
Identified by the controUer (178). the embedded data codes wm Identify 
the question corresponding to that trigger point. The controUer (178) 
will access the memory location address identified with the particular 
question and request the transfer of the particular user Input answer to 
that question. THis user Input Is mapped via some algorithm into an 
appropriate audio segment 

For example, suppose the narrator in the conventional program is 
currently discussing the size of the kitchen. At the next trigger point 
the controUer wiU access the memory location corresponding to the 
totchen Size question Usted above. Assuming the user selected response 
A 10*20". the controUer may select and forward the foUowing audio 
response to the audio switch. "Since you have a very smaU kitchen, may I 
suggest that you not put a dining room table In your kitchen." At the 
conclusion of this segment the controUer directs the controUer (178) to 
switch back to the standard audio. At the next trigger point a simUar 
sequence as described above wUI occur, fo this manner, personahzed 
Interactivity is provided at unexpected times to the subscriber. 

For unauthorized subscribers, the Interactive program box (600) 
acts similarly to a cable set top converter box. However, the Interactive 
program box (600) Is not required for rccepUon of the conventional 
program by nonlnteractive subscribers. If the subscriber has the box 
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(600) but is not authozlzed to Interact with the program for some reason, 
the video demodulator (616) simply demodulates the signal and the 
standard audio and video are transferred through the audio switch to the 
TV set (186) for presentation for the subscriber. 
5 Another application of the memory capability of the present 

invention is targeted marketing of subscribers. Iii this application, the 
interactive program box processor compiles and stores information 
including the Interactive program box number associated with the 
subscriber, time and date of the program viewing, the list of subscriber 

10 selections made by the stibscriber during the program, and the program 
ID (contained in the program codes broadcast with the program). This 
data is preferably stored together in data packets. FVom this data, a data 
code can be generated which provides an overall profile of the 
subscribers purchasing desires. This data code can either be sent via 

15 modem and over telephone lines to a retail location or displayed or 
printed out at the subscriber home. This code can then be used by the 
retail stores to offer unique purchasing price reductions and other 
options to the subscribers. 

While in the preferred embodiment the system has been described 

20 as a simulcast system whereby both an interactive program and a 
noninteractive program are broadcast to subscribers, the system does not 
require tbe program to be noninteractive. In other words, the system is 
not limited to providing the dual functional programs but can simply 
provide interactive programming, not receivable without the interactive 

25 program boxes (6(X)), using the same elements described above. 

The above systems of the present invention can be combined with 
those described in earlier Freeman U.S. Patent Nos. 4,847,700, 
4.507,680. 4.573.072. 4.602.279. 4.364. 925. and 4.264. 924 and in 
commonly copending U.S. Patent Application Serial No. 07/797,298. filed 

30 November 25. 1991. restalting with home viewers equipped with a more 
sophisticated decoder box to receive more enhanced interactivity such as 
instant replay, targeted advertising, etc., through second or third 
transmitted video signals, all of v^ch can be provided with the 
personalized audio and graphics in the manner described above. 
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Using the foregoing embodiments, methods and processes, the 
ACTV simulcast system maximizes personalized attention and 
interactlvlly to subscribers in their homes in real time using a low cost 
interactive system. Although the present Invention has been described in 
detail with respect to certain embodiments and examples, variations and 
modifications exist which are within the scope of the present invention 
as defined in the following claims. 
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1. A fiilly Interactive system for receiving an interactive program signal 
simulcasting an interactive program and a conventional television 
program, both programs provided with one conventional television video 
signal, to multiple subscriber locations permitting authorized interactive 
subscribers to interact with and receive different and personalized 
responses from the interactive program and noninteractive subscribers to 
view and listen to a conventional television program, the system 
comprising: 

at least one means for receiving the interactive program 
signal comprising a plurality of audio channels, graphics data, 
control data, a standard video signal and standard audio signal; 
at least one means for displaying the standard video signal; 
at least one interactive program means, connected to the 
receiving means and display means, comprising: 

a means for demodulating the standard video and 
standard audio signal; 

means for extracting, connected to the demodulating 
means, the plurality of audio channels and graphics and 
control data from the interactive program signal, wherein 
each audio channel comprises one or more audio segments; 

means for storing the graphics and control data, 
wherein the control data comprises branching codes and the 
graphics comprises graphics codes designating at least one 
interrogatory message: 

means for gathering at least one subscriber entiy; 

program selection means, operably connected to the 
first storage means and extracting means, for reading the 
control data, controlling operation of the Interactive program 
means, and selecting a personalized audio response 
comprising one or more audio segments, comprising: 

means for selecting one or more audio segments 

or graphics messages to form a personalized audio or 

graphics response, whereby the subscriber entry and 
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branchixig codes are used In each branch to formtOating 
ttit perscmaUzed audio or graphics response: 

a graphics generator, connected to the program 
selection means, for presenting graphics messages on the 
display means: and 

means for toteractiyely presenting the standard video 
and standard audio and the personalized audio response to 
the interacttve subscribers so that toteractive subscribers fully 
interact with the program to alter program content and 
Interact uniquely and differently from other interactive 
subscribers to receive personalized responses while 
nontoteracttve subscribers conttoue to receive a conventional 
television program. 

2. The fully toteractive ^tem of dalm 1 \vhereto the plurality of audio 
channels, standard video signal and standard audio signal, and graphics 
and control data of the totnactlve program signal are analog and 
embedded to a standard NTSC video signal and the extracttag means 
comprises means for removtog the plurality of audio channels and 
gr^hics and control data from the standard NTSC video signal. 

3. The fully toteractive system of daim 1 whereto the pluraUty of audio 
chaimels. standard video signal and standard audio signal, and graphics 
and control data are digitally encoded and combtoed to a composite 
digital toteractive signal and the receiving means further comprises: 

means for demultiplextog the plurality of audio channels, 
standard video signal and standard audio signal, and data from the 
composite digital toteractive signaL 

4. The fuUy toteractive system of claim 1 whereto the toteractive 
program is a recorded program to be received at a later time. 

5. The fuUy toteractive system of claim 1 whereto the program 
selection means directs the storage means to store one or more previous 
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subscriber entries and wherein the program selection means comprises 
means for processing the previous subscriber entries along with the 
current subscriber entry so that a selection is based on prior subscriber 
entries, the current subscriber entry and the codes in formulating the 
5 personalized audio or graphics response. 

6. The fully interactive system of claim 1 wherein the program 
sdection means directs the storage means to store one or more previous 
subscriber entries and wherein the program selection means comprises 

10 means for processing the previous subscriber entries along with the 
current subscriber entiy for determining the next graphics message to 
overiay onto the video signal. 

7. The fully interactive system of claim 1 wherein the means for 
15 gathering a subscriber entiy compises: 

a remote contndler, comprising: 

means for entering subscriber entry; 

means for preparing a return signal to represent the 
subscriber entry; and 
20 means for transmitting the subscriber entry to the 

interactive program box. 

a The fully interactive system of dalm 1 ^diereln each channel of the 
idurality of channels comprises a sequence ot audio response segments 

25 separated in time, with each audio response segment concurrent with 
audio segments in other channels thereby forming an array of audio 
ree^nse segments and the program selection means comprises means 
for picking an audio response segment from the entire array of audio 
response segments so that the quantity of the selectable personalized 

30 audio response segments is greater than the quantity of the plurality of 
audio channels. 

9. An interactive program production ^tem used for creating a sln^e 
composite Interactive signal to multiple subscribers, the composite 
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interactive signal slmialcastlng both a conventional television program and 
a ftalty interactive program, both programs provided in a conventional 
television bandwidth to multiple subscribers, comprising: 

means for creating a conventional video signal comprising a 
standard audio and standard video; 

means for generating graphics data, the data being used for 
identifying graphics characters that arc to be overlaid on the video 
at predetermined points in time; 

means for providing a pluraliiy of audio chaxmels. the audio 
ch a nnel s comprising audio segments, the audio segments being of 
substantially equal length in time and related in content to each 
other; 

means for creating control data comprising codes, whereby 
the codes are used for initiating, controlling, and branching 
between the plurality of audio channels; 

means for combining the conventional video signal, control 
data and pliurallty of audio channels into a single composite 
interactive signal: and 

means for transmitting the single composite interactive signal 
to multiple subscribers. 

10. The interactive program production system of claim 9. the imlt 
further comprising: 

means for digitaUjr encoding the conventional video signal and 
plurality of audio r^h ^n nfls: and 

means for multiplexing the control and graphics data, the 
plurality of audio channels and the conventional video signal into a 
single composite digital interactive signal. 

11. The interactive program production system of claim 9 wherein the 
conventional video signal, graphics and control data and plurality of audio 
channels are analog and the combining means comprises means for 
embedding the graphics and control data, plurality of audio channels and 
conventional video signal into a standard NTSC video signal. 
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12. Tlie Interactive program production system of claim 9 \i^ereln the 
conventional video signal comprises a vertical blanking interval and the 
embedding means comprises means for Inserting the graphics and 
5 control data and plurality of audio channels Into the vertical blanking 
Interval of the conventional video signal. 

IS. The Interactive program production system of claim 9 wherein the 
tra n s mi ss io n means comprises a television broadcasting means. 

14. The Interactive program production i^tem of dalm 9 wherein the 
transmission means comprises a cable means. 

15. The interactive program production system of claim 9 wherein the 
1 5 t r a nsmissi on means comprises a satellite broadcast means. 

16. The interactive program production system of dalm 9 \«^erein the 
means for providing a plurality of audio channels comprises a means for 
Unking a sequence of audio response segments in each channel forming 

20 an array of stacked audio segments so that the quantity of selectable audio 
response segments is greater than the quantity of the plurality of audio 
channels. 

17. The interactive program production ^tem of claim 9 further 
25 comprising a means for recording the composite Interactive signal for 

„; later transmission. 

18. The interactive program production i^tem of claim 9 wherein the 
composite Interactive signal comprises a live program. 

30 

19. An Interactive program production system used for creating a single 
composite Interactive signal to multiple subscribers, the composite 
mteractive signal simulcasting both a conventional television program and 
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a fully Interactive program, both programs provided in a conventional 
television bandwidth to multiple subscribers, comprising: 

means for creating a conventional video signal comprising a 
standard audio and standard video: 

means for generating graphics data, the data being used for 
Identifying graphics characters that are to be overlaid on the video 
at predetermined points In time Including means for generating a 
first graphic interrogatory question to overlay the conventional 
video signal If the fully interactive program Is being simulcast: 

means for providing a plurality of graphic channels, the 
graphic channels comprising graphic segments responsive to the 
first graphic Interrogatory, the graphic segments being of 
substantially equal length In time and related in content to each 
other: 

means for creating control data comprising codes, whereby 
the codes are used for initiating, controlling, and branching 
between the plurality of audio and graphic channels: 

means for combining the conventional video signal, control 
data and plurality of graphic channels Into a single composite 
Interactive signal; and 

means for transmitting the single composite Interactive signal 
to multiple subscribers. 

20. The Interactive program production system of Claim 19 wherein 
the system Is used to simulcast a sporting event In a conventional and an 
Interactive format the means for generating a first graphic Interrogatory 
being used to enable selection by an Interactive subscriber of statistics for 
a particular player or team and the means for generating a plurality of 
graphic channels being used to provide the statistics selected In response 
to the Interrogatory question. 

21. The interactive program production system of Hnfm 19 further 
comprising: 
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means for int>vlding a plurality of audio channels, the audio 
channels comprising audio segments related in time and content to 
the plurality of graphic channels, the audio segments being of 
substantially equal length in time and related in content to each 
other: and wherein the combining means comprises means for 
combining the conventional video signal, control data and plurality 
of audio and graphic channels into a single composite interactive 
signal 

22. An interactive program production system used for creating a sln^e 
composite interactive signal to mulUple subscribers, the composite 
interactive signal simulcasting both a conventional television program and 
a fully interactive program, both programs provided in a conventional 
television bandwidth, to multiple subscribers, comprising: 

means for creating a conventional video signal comprising a 
standard audio and standard video: 

means for generating graphics data, the data being used for 
identifying graphics characters that are to be overlaid on the video 
at predetermined points in time including means for generating a 
first graphic interrogatory question to overlay the conventional 
video signal if the fully interactive program is being simtalcast: 

means for providing a plurality of graphic channels, the 
graphic chazmels comprising graphic segments responsive to the 
first graphic interrogatory, the graphic segments being of 
substantially equal length In time and related in content to each 
other: 

means for providing a plurality of audio channels, tiie audio 
channels comprising audio segments related in time and content to 
the plurality of graphic channels, the audio segments being of 
substantially equal length in time and related in content to each 

other: 

means for creating control data comprising codes, whereby 
the codes are used for initlatmg, controlling, and branching 
between the plurality of audio and graphic channels: 
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means for combining the conventional video signal, control 
data and plurality of audio and graphic channels into a sin^e 
composite interactive signal: and 

means for transmitting the sin^ composite interactive signal 
to multiple subscribers. 

23. The interactive program production system of Claim 22 wherein 
the system is used to simulcast a sporting event in a conventional and an 
Interactive format the means for generating a first graphic interrogatory 
being used to enable selection by an interactive subscriber of statistics for 
a particular phQ^ or team; the means for generating a plurality of graphic 
chaimels being used to provide the statistics selected in response to the 
interrogatory question: and the means for providing a pluraUty of audio 
chaimels being used to provide a voice response reading or commenting 
on the statistics chosen which can be the voice of the player or owner of 
the team whose statistics were requested in the response to the 
interrogatory questiorL 

24. An interactive program production ^tem used for creating a single 
composite interactive signal to multiple subscribers, the composite 
interactive signal sim u lcas t ing both a conventional television program and 
a fully interactive program, both programs provided in a conventional 
television bandwidth to multiple subscribers, comprising: 

means for creating a ccmventional video signal comprising a 
standard audio and standard video; 

means for generating graphics data, the data being used for 
identifying graphics characters that are to be overlaid on the video 
at predetermined points in time; 

means for providing a plurality of interactive video channels, 
the audio channels comprising video segments, the video segments 
being of substantially equal length in time and related in content to 
each other: 
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means for creating control data comprising codes, whereby 
the codes are used for initiating, controlling, and branching 
between the plurality of interactive video channels; 

means for combining the conventional video signal, control 
data and plurality of interactive video channels into a single 
composite interactive signal; and 

means for transmitting the sln^ composite interactive signal 
to multiple subscribers, each of the subscribers having a means to 
receive the conventional video signal and ait least one of the 
subscribers having means to receive the interactive video signal 
with the ability to switch between the plurality of video channels. 

25. The interactive program production system of Claim 24 wherein 
tile qrstem is used to simulcast a sporting event in a conventional and an 
Interactive format ix^ereln the means for goieratlng graphics data 
includes means for generating a first graphic interrogatory question to 
overlay the conventional video signal if the fully interactive program is 
being simulcast to enable selection by an interactive subscriber of 
different an^es of play, a dose-up of a particular player or team or a 
replay of the last play and the means for generating a plurality of 
interactive video channels is used to provide the video channel -vtiilch 
displays the sdectlon selected in response to the Interrogatory question. 

26. A method for offering a single composite interactive program signal 
for slmulcasttng an interactive program and a conventional program both 
within a conventional television bandwidth to multiple subscriber homes, 
the method conq)rlsing the steps of: 

packaging the single composite Interactive program signal 
comprising a plurality of audio channels, graphics data and control 
data and a standard video signal and standard audio signal, wherein 
the single composite interactive program signal comprises a 
conventional television program and an interactive program; 

transmitting the single composite interactive program signal 
to at least one receive site location; 
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receiving the single composite Interactive program signal; 

extracting the audio channels, graphics data and control data 
from the sin^e composite interactive program signal; 

storing the graphics data and control data, whereby the 
control data comprises branch codes: 

generating graphics characters, the graphics characters 
comprising at least one interrogatory message: 

overlaying the graphics characters onto portions of the 
standard video signal to obtain a resulting video signal; 

displaying the resulting video signal on a display monitor: 

gathering a subscriber entry to the graphics interrogatoxy 
message: 

branching between the pliuallly of audio channels to fonn a 
personalized audio response, whereby the subscriber entry and 
branching codes are used in formulating the personalized audio 
response: 

presenting the personalized response to the subscriber, 
whereby each subscriber Interacts with the interactive program 
individually and differently from other subscribers, vnth each 
subscriber receiving a unique personalized audio response. 

27. An interactive home entertainment and educational system for 
receiving a fully interactive program signal, the fiiUy interactive program 
signal transmitted in a single standard television broadcast bandwidth, 
the system comprising: 

at least one means for receiving the interactive program 
signal comprising a plurality of audio channels comprising audio 
segments, data and a video signal, the interactive program signal 
carried in a single standard television broadcast bandwidth: 
at least one means for displaying the video signal; 
at least one interactive program box. connected to the 
receiving and display means, comprising: 

means for eictracting. connected to the receiving 
means, the plurality of audio channels and data: 
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means for storing a plurallQr of graphics Interrogatory 
messages: 

means for gathering a subscriber entty; 

a program selection means, connected to the extracting 
means and storage means, for selecting a personalized 
graphics interrogatoty messages finom ^e plurality of graphics 
Interrogatory messages and an audio response comprising at 
least one personalized audio segment from the plurality of 
audio channels, comprising: 

means for processing previous subscriber entries 

along with the current subscriber entiy for determining 

a graphics interrogatory message; 

means for formulating a personalized audio 

response comprising at least one audio segment* 

i^erein the audio response is chosen as a function of a 

most current subscriber entry and at least one previous 

subscriber entry; 

a graphics generator, coxmected to the program 
selection means, for overlajring the selected graphics 
interrogatory message onto the video signal; and 

a means for presenting the personalized audio response 
to the subscribers thereby allowing each subscriber to fuUy 
interact imiquely and different)^ from other subscribers at 
other subscriber locations. 

28, The interactive home entertairunent and educational s)rstem of 
dalm 27 herein the plurality of audio channels, video signal, and data of 
the interactive program signal are analog and embedded in a standard 
NTSC video signal and the extracting means comprises means for 
removing the plurality of audio channels and graphics and control data 
from the standard NTSC video signal. 

29. The interactive home entertalrmient and educational system of 
claim 27 wherein the plurality of audio charmels, video signal, and data 
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arc dlgltaUy encoded and combined In a composite digital Interactive 
signal and the receiving means further comprises: 

means for demultiplexing the plurality of audio channels, 
video signal and data from the composite digital Interactive signal; 
5 and 

means for converting the demultlplexfed audio channels and 
the demultiplexed video signal into analog signals. 
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30. The Interactive home entertainment and educational system of 
claim 27 wherein each channel of the plurality of audio channels 
comprises a sequence of audio response segments separated in time, 
with each audio response segment concurrent with audio segments iii 
other channels thereby forming an anray of audio response segments and 
the program selection means comprises means for picking an audio 
response segment from the entire array of audio response segments so 
that the quantity of the selectable personalized audio response segments 
Is greater than the quantity of the plurality of audio channels. 
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